Image sensor

ABSTRACT

An image sensor comprising one or more processors and/or circuitry which functions as: a plurality of pixels each of which detects photons incident during a predetermined exposure period, counts a number of the photons, and outputs a first count value; a calculator that calculates a second count value per unit time based on the exposure period and the first count value; and a corrector that acquires a correction coefficient based on the second count value and corrects a detection error of the first count value using the correction coefficient, wherein the corrector acquires a larger value as the correction coefficient in a case where the second count value is a first value than in a case where the second count value is a second value which is smaller than the first value.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to an image sensor and more specificallyto an image sensor for counting the number of incident photons andoutputting a counted value.

Description of the Related Art

Conventionally, CCD and CMOS image sensors have been widely used inimage capturing apparatuses. A common method used in these image sensorsis to convert incident light during an exposure period into electriccharges by photodiodes and accumulate them, read out the charges as ananalog signal of current or voltage, and convert the analog signal intoa digital signal.

Meanwhile, in recent years, a photon counting type image sensor whichcounts the number of photons incident on a photodiode during an exposureperiod and outputs the count value as a signal value has been proposed.As means for realizing the photon counting method, for example, there isa method of using an avalanche photodiode and a counter circuit. When areverse bias voltage larger than the breakdown voltage is applied to theavalanche photodiode, carriers generated due to the incidence of aphoton cause avalanche multiplication phenomenon and a large current isgenerated. By counting the pulse signal generated based on this currentby a counter circuit, a signal value corresponding to the number ofphotons incident on the avalanche photodiode can be obtained.

In the photon counting type image sensor, the number of photons incidenton the photodiode is treated as a signal value as it is. Therefore,compared to CCD and CMOS image sensors, there is little influence of acircuit noise on a signal, and thus it is possible to capture imagesclearly even in weak light environments.

However, in the photon counting type image sensor, as the amount ofreceived light per unit time increases, photons are incident with aperiod shorter than the pulse width of the pulse signal, and a pluralityof pulse signals may be combined. This causes the count value counted bythe counter circuit to be smaller than an actual number of photons, anda linear signal value with respect to the received light amount cannotbe obtained. That is, the linearity characteristic deteriorates. As aresult, the image quality of the captured image deteriorates.

Accordingly, Japanese Patent Laid-Open No. 2014-81253 discloses a photoncounting photodetector having cumulative means for obtaining an outputvalue obtained by cumulatively adding pulse widths of a pulse signal inorder to suppress a drop in the count value. With this configuration, asignal which monotonically increases with respect to an amount ofreceived light can be obtained.

Further, Japanese Patent No. 5917160 discloses a method in whichdeterioration in linearity characteristics caused by incomplete transferof charges in a CMOS image sensor is compensated for using a gaincorrection value and an offset correction value corresponding to anexposure amount (that is, an amount of light received during an exposureperiod).

However, in the configuration described in Japanese Patent Laid-Open No.2014-81253, for example, when a plurality of photons are simultaneouslyincident on one avalanche photodiode, a correct signal value cannot beobtained. Therefore, the linearity with respect to the received lightamount is insufficient.

Further, in the correction method described in Japanese Patent No.5917160, the gain correction value and the offset correction value arechanged according to the exposure amount. However, in the photoncounting type image sensor, between the case of counting for a shorttime under high illuminance environment (a received light amount perunit time is large) and the case of counting for a long time under lowilluminance environment (a received light amount per unit time issmall), even if the exposure amounts are the same, the decreases incount value is larger in the former case. Therefore, it is insufficientto merely change the correction value in accordance with the exposureamount. Especially in low illuminance environment, overcorrection mayoccur and the image quality may be deteriorated.

SUMMARY OF THE INVENTION

The present invention has been made in consideration of the abovesituation, and suppresses deterioration of image quality caused by adifference in received light amount per unit time in an image capturingapparatus which counts the number of incident photons and outputs acount value as a signal value.

According to the present invention, provided is an image sensorcomprising one or more processors and/or circuitry which functions as: aplurality of pixels each of which detects photons incident during apredetermined exposure period, counts a number of the photons, andoutputs a first count value; a calculator that calculates a second countvalue per unit time based on the exposure period and the first countvalue; and a corrector that acquires a correction coefficient based onthe second count value and corrects a detection error of the first countvalue using the correction coefficient, wherein the corrector acquires alarger value as the correction coefficient in a case where the secondcount value is a first value than in a case where the second count valueis a second value which is smaller than the first value.

Furthermore, according to the present invention, provided is an imagesensor comprising one or more processors and/or circuitry whichfunctions as: a plurality of pixels each having a detector that detectsphotons, a counter that counts a number of the detected photons andoutputs a counted value, and a memory that holds the counted value, andoutputs the counted value held in the memory as a first count value eachtime a predetermined exposure period elapses; a calculator thatcalculates a second count value per unit time based on the exposureperiod and a difference value between the first count values output insequence; and a corrector that obtains a correction coefficient based onthe second count value and corrects a detection error of the differencevalue using the correction coefficient, wherein the corrector obtains alarger value as the correction coefficient in a case where the secondcount value is a first value than in a case where the second count valueis a second value which is smaller than the first value.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments (with reference to theattached drawings).

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate embodiments of the invention, andtogether with the description, serve to explain the principles of theinvention.

FIG. 1 is a block diagram showing an overall configuration of an imagecapturing apparatus according to a first embodiment of the presentinvention;

FIG. 2 is a diagram showing a configuration of an image sensor accordingto the first embodiment;

FIG. 3 is a circuit diagram showing an example of a configuration of aunit pixel according to the first embodiment;

FIG. 4 is a diagram for explaining an operation of a light receivingunit according to the first embodiment;

FIG. 5 is a schematic diagram showing a configuration of a pixel areaaccording to the first embodiment;

FIG. 6 is a schematic diagram showing an example of a chip layout of theimage sensor according to the first embodiment;

FIG. 7 is a drive timing chart of the image capturing apparatus when animage of one frame is captured according to the first embodiment;

FIG. 8 is a block diagram illustrating a configuration of a correctionunit according to the first embodiment;

FIG. 9 is a diagram showing a relationship between a count rate and anincident photon rate;

FIG. 10 is a diagram showing a relationship between an incident photonrate and a count rate;

FIG. 11 is a diagram showing an example of a gain correction coefficientaccording to the first embodiment;

FIG. 12 is a circuit diagram showing an example of a configuration of aunit pixel according to a modification of the first embodiment;

FIG. 13 is a block diagram illustrating a configuration of a correctionunit according to a second embodiment;

FIG. 14 is a diagram showing examples of gain correction coefficientsaccording to the second embodiment;

FIG. 15 is a flowchart of image shooting processing according to thesecond embodiment;

FIG. 16 is a block diagram illustrating a configuration of a correctionunit according to a third embodiment;

FIGS. 17A and 17B illustrate an offset correction method according tothe third embodiment;

FIG. 18 is a block diagram illustrating a configuration of a correctionunit according to a fourth embodiment;

FIGS. 19A and 19B illustrate an offset correction method according tothe fourth embodiment;

FIG. 20 is a diagram showing a configuration of an image sensoraccording to a fifth embodiment;

FIG. 21 is a schematic diagram showing an example of a chip layout ofthe image sensor according to the fifth embodiment;

FIG. 22 is a drive timing chart of the image capturing apparatus when animage of one frame is captured according to the fifth embodiment;

FIG. 23 is a block diagram showing a configuration of a signalprocessing block according to a sixth embodiment; and

FIG. 24 is a drive timing chart of the image capturing apparatus when animage of one frame is captured according to the sixth embodiment.

DESCRIPTION OF THE EMBODIMENTS

Exemplary embodiments of the present invention will be described indetail in accordance with the accompanying drawings.

First Embodiment

A first embodiment of the present invention will be described. FIG. 1 isa block diagram showing the overall configuration of an image capturingapparatus according to the first embodiment. In FIG. 1, an imaging lens102 is driven by a lens driving unit 103, and performs zoom control,focus control, aperture control, and the like to form an optical imageof a subject on an image sensor 100.

The image sensor 100 has a configuration capable of counting the numberof incident photons and outputting it as a signal value, receives theoptical image of the subject formed by the imaging lens 102 as incidentlight and outputs an image signal. The configuration of the image sensor100 will be described later in detail. A signal processing unit 101performs various kinds of signal processing including rearrangement onthe image signal output from the image sensor 100. An overallcontrol/arithmetic unit 104 performs various arithmetic processing andcontrols the entire image capturing apparatus.

A memory unit 105 is used for temporarily storing image data, and adisplay unit 106 displays various information and captured images. Arecording unit 107 has a structure in which a semiconductor memory andthe like can be attached and detached, and performs recording or readingof image data. An operation unit 108 includes buttons, dials, and thelike, and receives inputs caused by operation of the operation unit 108by the user. If the display unit 106 is a touch panel, the touch panelis also included in the operation unit 108. A photometry unit 109includes an image sensor, such as a CCD or CMOS sensor (not shown), forphotometry, receives light passing through the imaging lens 102 via amovable mirror (not shown) or the like, and calculates luminance of thesubject for each of a plurality of photometry areas.

FIG. 2 is a diagram showing an overall configuration of the image sensor100. The image sensor 100 includes a pixel area 200, a vertical controlcircuit 202, a horizontal control circuit 203, a timing generator (TG)204, a correction unit 207, and a digital output unit 208.

In the pixel area 200, unit pixels 201 each having an avalanchephotodiode and a counter circuit are arranged in a matrix as describedlater. Here, in order to simplify the explanation, an array of 4×4 unitpixels 201 is shown, but in practice more pixels are arranged. The unitpixel 201 can count the number of incident photons and output it as adigital signal value. Details of the unit pixel 201 will be describedwith reference to FIG. 3.

The vertical control circuit 202 selects the unit pixels 201 in thepixel area 200 on a row-by-row basis using switches 205. In addition,the vertical control circuit 202 sends control signals to the unitpixels 201 in the pixel area 200 on a row-by-row basis via wires (notshown). Details of this control signal will also be described later withreference to FIG. 3.

The horizontal control circuit 203 selects the unit pixels 201 in thepixel area 200 by switches 206 on a column-by-column basis. A pixelsignal of the unit pixel 201 selected by the combination of the verticalcontrol circuit 202 and the horizontal control circuit 203 is output tothe correction unit 207.

The correction unit 207 performs a correction process (image processing)for suppressing deterioration in image quality caused by a difference inreceived light amount per unit time on the pixel signal output from eachpixel unit 201. The correction unit 207 also performs a correctionprocess of matching the black levels of the pixel signals with areference value. Details of these correction processes will be describedlater with reference to FIG. 8.

The digital output unit 208 outputs the pixel signals corrected (imageprocessed) by the correction unit 207 to the outside of the image sensor100. The TG 204 sends control signals for outputting the pixel signal ofeach pixel to the vertical control circuit 202 and the horizontalcontrol circuit 203. It should be noted that the TG 204 also sendscontrol signals to the correction unit 207 and the digital output unit208 via wires (not shown).

FIG. 3 is a diagram showing a configuration of the unit pixel 201. Theunit pixel 201 is roughly divided into a light receiving unit 301 and acounting unit 302.

The light receiving unit 301 includes a photodiode (PD) 303, a quenchingresistor 304, and an inverting buffer 305. The PD 303 is an avalanchephotodiode, and a reverse bias voltage Vbias larger than the breakdownvoltage is applied through the quenching resistor 304 and operates inthe Geiger mode. Therefore, when a photon enters the PD 303, anavalanche multiplication phenomenon is caused and an avalanche currentis generated. The quenching resistor 304 is a resistor element forstopping the avalanche multiplication phenomenon of the PD 303. Theinverting buffer 305 is provided for taking a voltage change caused bythe avalanche multiplication phenomenon as a pulse signal (hereinafterreferred to as “PLS signal”).

The operation of the light receiving unit 301 will be described withreference to FIG. 4. FIG. 4 is a diagram showing transition of a cathodeterminal voltage Vout and the PLS signal output from the invertingbuffer 305 when photons are incident on the PD 303 with respect to time.Here, the cathode terminal voltage Vout is also the magnitude of thereverse bias voltage applied to the PD 303.

At time t401, the reverse bias voltage Vbias larger than a breakdownvoltage Vbr is applied to the PD 303 and the PD 303 operates in theGeiger mode. In this state, when a photon enters the PD 303 at timet402, carriers generated by the PD 303 cause an avalanche multiplicationphenomenon, and an avalanche current is generated. Due to this avalanchecurrent, the cathode terminal voltage Vout of the PD 303 connected tothe quenching resistor 304 starts to decrease. When the cathode terminalvoltage Vout falls below the breakdown voltage Vbr at time t404, theavalanche multiplication phenomenon stops. Then, the cathode terminalvoltage Vout starts to rise as it is recharged by a power source towhich the reverse bias voltage Vbias is applied via the quenchingresistor 304. When recharging is completed at time t406, the cathodeterminal voltage Vout returns to the reverse bias voltage Vbias again.Here, the time required to recharge the cathode terminal voltage Voutdepends on a resistance value of the quenching resistor 304 and aparasitic capacitance.

Assuming that a threshold value at which the PLS signal output from theinverting buffer 305 switches from L to H and from H to L is Vth, thePLS signal becomes H between times t403 and t405 as shown in FIG. 4.Therefore, one PLS signal is output from the light receiving unit 301 byone photon incidence. Here, a pulse width of the PLS signal generated attimes t403 to t405 is denoted by ΔTp.

Returning to the description of FIG. 3, the counting unit 302 includes acounter circuit 306 and a pixel memory 307. To the counter circuit 306,the PLS signal generated in response to the incidence of photons by thelight receiving unit 301 is input, and the number of times that the PLSsignal changes from L to H is counted as the number of pulses. An enablesignal (hereinafter referred to as “PEN signal”) and a reset signal(hereinafter referred to as “PRES signal”) are supplied from thevertical control circuit 202 to the counter circuit 306. When the PLSsignal changes from L to H while the PEN signal supplied to the countercircuit 306 is at the H level, the count value increases by one. In thestate where the PEN signal is at the L level, even when the PLS signalchanges from L to H, the count value does not increase and the currentcount value is retained. When the PRES signal supplied to the countercircuit 306 becomes the H level, the count value of the counter circuit306 is reset to 0.

The pixel memory 307 is used for temporarily holding a count valuecounted by the counter circuit 306 as a digital pixel signal while pixelsignals are read out to the correction unit 207. A latch signal(hereinafter referred to as “PLAT signal”) is supplied from the verticalcontrol circuit 202 to the pixel memory 307. When the PLAT signalchanges from L to H, the pixel memory 307 captures and holds the countvalue of the counter circuit 306 at that time as a digital pixel signal.Thereafter, the value counted by the counter circuit 306 is referred toas the count value, and the value held by the pixel memory 307 isreferred to as a pixel signal in order to distinguish between thesesignals.

The pixel signal held in the pixel memory 307 of the unit pixel 201selected by the combination of the vertical control circuit 202 and thehorizontal control circuit 203 is output to the correction unit 207.

In the first embodiment, the PEN signal, the PRES signal, and the PLATsignal supplied from the vertical control circuit 202 are described asbeing controlled simultaneously for all the unit pixels 201 in the pixelarea 200, however, the timing of controlling the unit pixels 201 bythese signals may be controlled on a row-by-row basis.

As shown in FIG. 5, a partial area of the pixel area 200 is composed ofan optical black pixel area (hereinafter referred to as “OB area”) 501and an open pixel region 502. The light receiving unit 301 of the unitpixel 201 (OB pixel) arranged in the OB area 501 is shielded by a metallight shielding layer or the like (not shown), and light is not incidenton the PD 303. The pixel signal output from each unit pixel 201 in theOB area 501 is used for correction to adjust the black level of thepixel signal to the reference value by the correction unit 207. On theother hand, the light receiving unit 301 of the unit pixel 201 arrangedin the open pixel region 502 has an aperture (not shown) so that lightenters the PD 303. From each unit pixel 201 in the open pixel region502, a pixel signal corresponding to the received light amount isoutput.

In FIG. 5, the OB area 501 is arranged in the upper side of the pixelarea 200, however, the position of the OB area 501 may be arbitrarilyselected, such as the left side and the right side of the pixel area200.

FIG. 6 shows an example of a chip layout of the image sensor 100. Theimage sensor 100 has a structure in which a light receiving unitsubstrate 601 and a counting unit substrate 602 are stacked. Wirings ofthese substrates are electrically connected by using silicon penetratingelectrodes or the like. In the light receiving unit substrate 601, thelight receiving units 301 of the unit pixels 201 are arranged in amatrix. In the counting unit substrate 602, the counting units 302 ofthe unit pixels 201 are arranged in a matrix. In addition, the verticalcontrol circuit 202, the horizontal control circuit 203, the TG 204, thecorrection unit 207, and the digital output unit 208 are also arrangedon the counting unit substrate 602. As shown in FIG. 5, by forming thelight receiving unit 301 and the counting unit 302 on separatesubstrates, the area of the light receiving unit 301 can be secured,whereby a decrease in aperture ratio of the light receiving unit 301 canbe prevented. Note that the image sensor 100 may be formed on a singlesubstrate instead of forming on the stacked substrates.

FIG. 7 is a drive timing chart of the image capturing apparatus whenobtaining an image of one frame in the first embodiment. When a shootingstart signal START changes from L to H at time t701, the reverse biasvoltage Vbias is supplied to the light receiving unit 301 of each unitpixel 201. Then, a reverse bias voltage larger than the breakdownvoltage is applied to the PD 303, and the PD 303 starts to operate inthe Geiger mode. As a result, the PLS signal corresponding to the numberof incident photons is output from the light receiving unit 301. COUNTshows an example of the count value counted by the counter circuit 306of an arbitrary unit pixel 201. Cmax is the maximum value that can becounted by the counter circuit 306.

At the time t701, the PRES signal is H, and the count value of thecounter circuit 306 of each unit pixel 201 is reset to 0.

At time t702, the PRES signal becomes L, and resetting of the countercircuit 306 of each unit pixel 201 ends. At the same time, the PENsignal becomes H, and the counter circuit 306 of each unit pixel 201 isenabled. Therefore, in the counter circuit 306 of each unit pixel 201,the count value increases in accordance with the input PLS signal.Accordingly, the exposure period of the image capturing apparatusstarts. This exposure period continues until the PEN signal goes to L attime t703. T denotes the length of the exposure period from time t702 totime t703.

When the exposure period T ends at time t703, the PEN signal becomes L.As a result, the counter circuit 306 of each unit pixel 201 is disabled,and even if the PLS signal is input to the counter circuit 306, thecount value does not increase. Also, the supply of the reverse biasvoltage Vbias to the light receiving unit 301 is stopped, and the lightreceiving unit 301 stops outputting the PLS signal. Then, at time t704,the PLAT signal supplied to each unit pixel 201 from the verticalcontrol circuit 202 simultaneously changes from L to H. As a result, thecount value of the counter circuit 306 of each unit pixel 201 is held inthe pixel memory 307 as a pixel signal. After that, the PLAT signalreturns to L. Further, when the pixel signal is held in the pixel memory307, the PRES signal goes to H immediately, and the count value of thecounter circuit 306 of each unit pixel 201 is reset to 0.

During times t705 to t706, a VCLK signal is supplied from the TG 204 tothe vertical control circuit 202. Each time the VCLK signal goes to H,the switches 205 in each row turn on in turn and the vertical controlcircuit 202 selects the unit pixels 201 in the pixel area 200 row byrow. When an arbitrary row is selected, an HCLK signal is supplied fromthe TG 204 to the horizontal control circuit 203, and the switches 206in each column are sequentially turned on. As a result, the pixelsignals held in the pixel memory 307 of the unit pixels 201 of theselected row are sequentially output to the correction unit 207.

Thereafter, the pixel signals corrected by the correction unit 207 aresequentially output to the outside of the image sensor 100 via thedigital output unit 208.

FIG. 8 shows a configuration of the correction unit 207. The correctionunit 207 includes a count rate calculation unit 801, a gain correctionunit 802, and a black level correction unit 803. The count ratecalculation unit 801 and the gain correction unit 802 correct imagequality deterioration caused by a difference in received light amountper unit time. The black level correction unit 803 removes the pixelsignal component (dark current component) that is inappropriatelyincreased since the counter circuit 306 has counted the PLS signaloutput from the light receiving unit 301 corresponding to the darkcurrent generated in the PD 303 of each unit pixel 201 by performingoffset correction.

Here, a method of correcting image quality deterioration (error) causedby a difference in received light amount per unit time will be describedwith reference to FIGS. 9 and 10.

FIG. 9 shows the relationship between an incident photon rate of anarbitrary unit pixel 201 and a count rate of the counter circuit 306.Here, the incident photon rate is the number of photons incident on thelight receiving unit 301 per unit time, which is proportional to thereceived light amount per unit time. The count rate is an increment perunit time of the count value of the counter circuit 306. Ideally, thecount rate and the incident photon rate are proportional as shown by theideal value. However, in a state where the incident photon rate is high,that is, in a state in which the amount of received light per unit timeis large, a new photon is incident before the PLS signal generated bythe incidence of one photon returns from H to L. As a result, aplurality of PLS signals are connected, and the count value counted bythe counter circuit 306 becomes lower than the actual number of photons.Therefore, as the incident photon rate increases, the count rate becomeslower than the actual value, as shown by 901 in FIG. 9.

FIG. 10 shows the relationship between the incident photon rate and thecount rate. FIG. 10 shows a graph in which the vertical axis and thehorizontal axis of the graph of FIG. 9 are exchanged. As shown in FIG.10, it is possible to obtain a pixel signal that is linear with respectto the amount of received light by performing gain correction on thepixel signal so that the pixel signal becomes an ideal value asindicated by an arrow 1001 in accordance with the count rate of eachunit pixel 201. Here, in the area indicated by slanted lines where theincident photon rate is Pmax or more, the count rate starts to decreaseas the incident photon rate increases. Therefore, in a case where theincident photon rate is Pmax or more, the true incident photon ratecannot be calculated from the count rate. For this reason, photometryprocessing is performed with the photometry unit 109 before imageshooting, and if the overall control/arithmetic unit 104 determines fromthe photometry result that a high-luminance subject whose incidentphoton rate is greater than or equal to Pmax exists in the scene, imageshooting is performed after controlling the aperture of the imaging lens102 so that the incident photon rate becomes less than Pmax.Alternatively, as described in Japanese Patent Laid-Open No. 2014-81253,accumulation means for cumulatively adding the pulse width of the PLSsignal output from the light receiving portion may be provided in eachpixel so that a signal monotonically increasing with respect to theamount of received light is output from each pixel.

Next, specific processing of each block in FIG. 8 will be described. Inthe count rate calculation unit 801, the pixel signals output from theunit pixels 201 are sequentially input. Then, by dividing the pixelsignal by the exposure period T, the count rate which is the incrementof the count value per unit time is calculated.

The gain correction unit 802 performs gain correction on the pixelsignal from the unit pixel 201 to be corrected based on the count ratecalculated by the count rate calculation unit 801. Let x be the pixelsignal output from each unit pixel 201 and input to the correction unit207 and let y the pixel signal after gain correction, then the pixelsignal y after gain correction can be expressed by Equation (1).

y=α(r)×x   (1)

Here, α(r) is a gain correction coefficient, r is the count ratecalculated in the count rate calculation unit 801.

FIG. 11 shows an example of the gain correction coefficient α(r). Asshown in the FIG. 11, the gain correction coefficient α(r) takesdifferent values depending on the count rater calculated by the countrate calculation unit 801. The gain correction coefficient α(r) is again correction amount necessary for making the count rate in FIG. 10 anideal value.

The gain correction coefficients α(r) may be stored as a correctiontable corresponding to count rates r in the gain correction unit 802.Alternatively, an function of count rate r for approximating the gaincorrection coefficient α(r) may be stored, and the gain correctioncoefficient α(r) may be calculated according to the count rate of eachunit pixel 201.

The black level correction unit 803 receives the pixel signalgain-corrected by the gain correction unit 802, and removes the darkcurrent component from the pixel signal by performing offset correction.Specifically, the dark current component is calculated by integratingthe pixel signals of the OB area 501, and calculating the average value.Then, the average value is subtracted from the pixel signal of each unitpixel 201 in the open pixel region 502, thereby removing the darkcurrent component.

The pixel signals corrected by the black level correction unit 803 aresequentially sent to the digital output unit 208 and output to theoutside of the image sensor 100.

The gain correction by the gain correction unit 802 is performed beforethe offset correction by the black level correction unit 803. This isbecause the pulse signal caused by the dark current in the lightreceiving unit 301 and the pulse signal generated by the incidence ofthe photon are connected, and therefore, both the pixel signal componentdue to the photons and the dark current component of the count valuecounted by the counter circuit 306 become smaller than the correctvalues. Therefore, in the gain correction unit 802, gain correction isperformed so that the pixel signal component corresponding to the actualnumber of incident photons and the dark current component of the countvalue are obtained, and then the average pixel signal of the OB area 501is subtracted from each pixel signal by the black level correction unit803, thereby suitably removing the dark current component.

Note that if the dark current component is sufficiently smaller than thepixel signal component corresponding to the photons, the gain correctionunit 802 may perform the gain correction after the offset correction isperformed by the black level correction unit 803.

Thus, by performing the correction processes in the correction unit 207as described above, image quality deterioration caused by a differencein received light amount per unit time can be suppressed. Note that itis possible to modify the configuration such that the correctionprocesses performed in the correction unit 207 may be performed in thesignal processing unit 101 or the overall control/arithmetic unit 104.

Modification

In the unit pixel 201 shown in FIG. 3, the avalanche multiplication isstopped by using the quenching resistor 304. Alternatively, a resistancecomponent of the MOS transistor may be used as the quenching resistance.FIG. 12 shows a configuration of a unit pixel 201 according to amodification of the first embodiment. FIG. 12 shows a configuration ofthe unit pixel 201 corresponding to the configuration shown in FIG. 3,and the same reference numerals as in FIG. 3 are given to the sameconstituents and description thereof is omitted.

FIG. 12 shows the configuration when the resistance component betweenthe drain and the source of a MOS transistor 1201 is utilized as aquench resistance. In this configuration, the time required forrecharging between times t404 to t406 in FIG. 4 can be changed bychanging the resistance value between the drain and the source bychanging a gate voltage Vqnc of the MOS transistor 1201. For example,when the MOS transistor 1201 is turned on by making the gate voltageVqnc equal to or higher than the gate threshold voltage, the resistancevalue between the drain and the source decreases. As a result, the timerequired for recharging is shortened, and a pulse width ΔTp of the PLSsignal is shortened. Therefore, when a plurality of photons aresuccessively incident on the light receiving portion, it is possible toreduce the probability that pulses generated by the respective photonsare connected.

As a result, it is possible to reduce the decrease of the output countrate in a case where the incident photon rate shown in FIG. 9 is high.In this case, by changing the gain correction coefficient α(r) to bemultiplied to the pixel signal by the gain correction unit 802 inaccordance with the pulse width ΔTp, excessive correction can besuppressed. Therefore, image quality deterioration caused by adifference in received light amount per unit time can be suitablysuppressed.

Second Embodiment

Next, a second embodiment of the present invention will be described. Inthe first embodiment described above, the count rate is calculated foreach pixel and correction is performed using the gain correctioncoefficient corresponding to the count rate. On the other hand, in thesecond embodiment, a gain correction coefficient corresponding to apixel signal is selected based on a photometry result of the photometryunit 109.

FIG. 13 is a block diagram showing a configuration of the correctionunit 207 in the second embodiment, which is used in place of thecorrection unit 207 shown in FIG. 8 described in the first embodiment.It is to be noted that the same reference numerals are given to the sameconfigurations as those in FIG. 8, and a description thereof will beomitted. In addition, since the configuration other than the correctionunit 207 is the same as that of the first embodiment described above,description thereof is omitted.

A correction coefficient selection unit 1301 acquires the photometryresult of the photometry performed by the photometry unit 109, andselects a gain correction coefficient corresponding to the pixel signalbased on the photometry result. Specifically, one of the gain correctioncoefficients indicated by γ1(x) to γ3(x) in FIG. 14 is selected based onthe maximum luminance value among the luminance values of subjectsmeasured for respective photometry areas of the photometry unit 109.Here, x indicates a pixel signal input to a gain correction unit 1302,and γ1(x) to γ3(x) are gain correction coefficients corresponding to thepixel signal.

If the maximum luminance value measured by the photometry unit 109 islower than a predetermined first threshold value, the gain correctioncoefficients γ1(x) are selected, and if the maximum luminance value isequal to or larger than a predetermined second threshold value, the gaincorrection coefficients γ3(x) are selected. Note that the firstthreshold value is smaller than the second threshold value. Further, inthe case where the maximum luminance value is between the first andsecond thresholds, the gain correction coefficients γ2(x) are selected.That is, for the same pixel signals, as the maximum luminance valuemeasured by the photometry unit 109 is higher, a gain correctioncoefficients whose gain correction amount is larger is selected. This isbecause the higher the maximum luminance value is, the greater thereceived light amount per unit time at the time of shooting increases,and the greater the influence of the drop of the pixel signal caused bythe pulse signals from the light receiving unit being connected.

In the gain correction unit 1302, the pixel signal x is multiplied bythe correction coefficient selected by the correction coefficientselection unit 1301 as in the following Equation (2).

y=γ(x)×x   (2)

Here, y is a pixel signal after the gain correction, and γ(x) is a gaincorrection coefficient selected based on the photometry result and thepixel signal by the correction coefficient selection unit 1301.

Similarly to the first embodiment, the pixel signal undergone the gaincorrection by the gain correction unit 1302 is subjected to the offsetcorrection by the black level correction unit 803, and the correctedpixel signal is output to the digital output unit.

FIG. 15 shows a flowchart of automatic exposure image shootingprocessing in the second embodiment. When image shooting is started, instep S1501, photometry processing is performed in the photometry unit109. Specifically, the luminance of a subject is measured for eachphotometry area of the photometry unit 109.

In step S1502, the overall control/arithmetic unit 104 sets the shootingconditions such as an aperture and an exposure period according to thephotometry result of the photometry unit 109. Then, in step S1503, thecorrection coefficient selection unit 1301 selects the gain correctioncoefficients. Here, as described above with reference to FIG. 14, gaincorrection coefficients whose gain correction amount is larger as themaximum luminance value measured by the photometry unit 109 is higher isselected.

In step S1504, image shooting is performed by the image sensor 100 underthe shooting conditions set in step S1502. Specifically, the drivingshown in FIG. 7 is performed, and pixel signals are sequentially outputfrom the pixel area 200 to the correction unit 207.

In step S1505, correction processes are performed on the pixel signalssequentially output from the pixel area 200 in the correction unit 207.In the gain correction unit 1302, among the gain correction coefficientsselected by the correction coefficient selection unit 1301, the pixelsignal is multiplied by a gain correction coefficient selected accordingto the pixel signal. Thus, it is possible to compensate for a decreasein pixel signal caused by pulse signals from the light receiving unitbeing connected. Thereafter, the pixel signals undergone the offsetcorrection by the black level correction unit 803 are output from theimage sensor 100, and stored as image data by the recording unit 107 instep S1506.

With the correction processes in the correction unit 207 as describedabove, it is possible to suppress image quality deterioration caused bya difference in received light amount per unit time.

Third Embodiment

Next, a third embodiment of the present invention will be described. Inthe third embodiment, a configuration for changing a correction amountof a black level correction unit based on a received light amount perunit time will be described.

In a pixel having a large received light amount per unit time, anavalanche multiplication phenomenon frequently occurs and a largecurrent flows through the quenching resistor 304, so that more heat isgenerated at the quenching resistor 304. Accordingly, the temperature ofthe PD 303 in the vicinity of the quenching resistor 304 rises and theamount of dark current increases. As a result, a difference in darkcurrent amount is caused between a pixel (OB pixel) in the OB regionwhere light is not incident and a pixel whose received light amount perunit time is large. Therefore, in the third embodiment, a correctionamount of a black level correction unit is changed according to areceived light amount per unit time.

FIG. 16 is a block diagram showing a configuration of the correctionunit 207 in the third embodiment, which is used in place of thecorrection unit 207 shown in FIG. 8 described in the first embodiment.It is to be noted that the same reference numerals are given to the sameconfigurations as those in FIG. 8, and a description thereof will beomitted. In addition, since the configuration other than the correctionunit 207 is the same as that of the first embodiment described above,description thereof is omitted.

A spatial filter processing unit 1601 performs a filtering process on acount rate for each pixel calculated by the count rate calculation unit801. For example, an average count rate which is the average value ofthe count rates of a target pixel and its surrounding eight pixels iscalculated. As the filtering process, a weighted average process may beperformed.

When the count rates of pixels to be read later than the target pixel isto be averaged, a holding memory (not shown) is provided before thecorrection unit 207 to temporarily hold the pixel signals and thenfiltering process may be performed by the spatial filter processing unit1601. A pixel signal output from the gain correction unit 802 is inputto a black level correction unit 1602, and offset correction representedby following Equation (3) is performed.

z=y−β(r _(ave))×drk   (3)

Here, z is a pixel signal after offset correction, y is a pixel signalinput to the black level correction unit 1602, drk is an average pixelsignal in the OB area, r_(ave) is an average count rate calculated bythe spatial filter processing unit 1601, and β(r_(ave)) is an offsetcorrection coefficient, which is a value corresponding to the averagecount rate r_(ave).

Details of the offset correction coefficient β(r_(ave)) will bedescribed with reference to FIGS. 17A and 17B. FIG. 17A shows an examplein which count rates of pixels in an arbitrary column in the pixel areaare calculated. As shown in 1701, the count rate is high in a pixelwhose received light amount per unit time is large. Further, in thevicinity of a pixel with a high count rate, the temperature increasesdue to the heat generated by the avalanche current as shown by thetemperature indicated by the solid line. The average count rateindicated by a broken line is a value obtained by performing anaveraging process on the count rate of each pixel by the spatial filterprocessing unit 1601.

FIG. 17B is an example of the dark current amounts of the pixelscorresponding to FIG. 17A. In the pixels near the position indicated by1702, since the temperature is high, the amount of dark current islarger than in the pixels in the OB area. β(r_(ave)) shown by a solidline indicates offset correction coefficients for multiplying theaverage pixel signals of the OB area. As shown in FIG. 17B, the offsetcorrection coefficient β(r_(ave)) has a larger value as the averagecount rate shown in FIG. 17A increases. In this manner, even if theamount of dark current differs between pixels in the OB area and pixelswhose received light amounts per unit times are large, it is possible toappropriately remove the dark current component from the pixel signal.

In a case where the pixel area 200 is covered by a Bayer array colorfilter, the spatial filter processing unit 1601 may perform theaveraging process separately for pixels of the same color. Further, anoffset correction coefficient corresponding to the count rate calculatedby the count rate calculation unit 801 may be used without providing thespatial filter processing unit 1601.

According to the third embodiment as described above, it is possible toappropriately eliminate the dark current component from the pixel signalof the pixel whose received light amount per unit time is large, and itis possible to suppress image quality deterioration caused by adifference in received light amount per unit time.

Fourth Embodiment

Next, a fourth embodiment of the present invention will be described. Inthe case of shooting images continuously, such as a case of shooting amoving image or the like, when a high luminance subject moves, a countrate of a pixel at the position where the subject was located decreases,and a decrease in temperature is much slower than the count rate.Accordingly, the dark current amount of each pixel gently changes inaccordance with the change in temperature. In such a case as well, aconfiguration capable of suitably correcting the dark current componentwill be described.

FIG. 18 is a block diagram showing a configuration of the correctionunit 207 in the fourth embodiment, which is used in place of thecorrection unit 207 shown in FIG. 8 described in the first embodiment.It is to be noted that the same reference numerals are given to the sameconfigurations as those in FIG. 8, and a description thereof will beomitted. In addition, since the configuration other than the correctionunit 207 is the same as that of the first embodiment described above,description thereof is omitted.

A temporal filter processing unit 1801 filters the count rate calculatedby the count rate calculation unit 801 over a plurality of frames. Forexample, for each pixel, the average count rate of five latest framesincluding the current frame is calculated. In addition, the temporalfilter processing unit 1801 includes a holding memory (not shown) forholding the average count rate of each pixel.

The pixel signal output from the gain correction unit 802 is input to ablack level correction unit 1802, and the black level is corrected usingthe offset correction coefficient β corresponding to the average countrate calculated by the temporal filter processing unit 1801. Since thiscorrection method is the same as the method described with reference tothe equation (3) in the third embodiment, explanation thereof isomitted.

FIG. 19A shows an example in which a count rate of an arbitrary pixel inthe pixel area is calculated for each frame. As shown in a frame 1901,even when the count rate of the target pixel is lowered due to themovement of the high luminance subject, for example, the temperature ofthe target pixel indicated by the solid line in the figure does notdecrease immediately but decreases gently. Therefore, as shown in FIG.19B, the dark current of each frame gradually changes according to thetemperature. The average count rate indicated by a broken line is avalue obtained by averaging the count rates of the target pixel overplural frames by the temporal filter processing unit 1801.

FIG. 19B is an example of the dark current amount over framescorresponding to FIG. 19A. In the frame 1902, although the count ratehas decreased, since the temperature of the pixel is still high, thedark current amount remains high comparing to the count rate. β(r_(ave))indicated by the solid line is an offset correction coefficient formultiplying the average pixel signal of the OB area in Equation (3). Theoffset correction coefficient β(r_(ave)) for a pixel has a larger valueas the average count rate of the pixel shown in FIG. 17A is higher. Byperforming offset correction by using the value obtained by multiplyingthe average pixel signal of the OB area by the offset correctioncoefficient, it is possible to remove the dark current component fromthe pixel signal.

According to the fourth embodiment as described above, it is possible toappropriately eliminate the dark current component from a pixel signaleven if a received light amount per unit time has largely changed duringshooting a moving image or the like.

Fifth Embodiment

Next, a fifth embodiment of the present invention will be described. Inthe first embodiment as described above, since the count rate iscalculated based on pixel signals output after the end of the exposureperiod and correction is performed, in a case where luminance of asubject changes during the exposure period, it is not possible to obtaina correct count rate. In consideration of the above defect, in the fifthembodiment, a configuration capable of handling a case where luminanceof a subject changes during the exposure period will be described.

FIG. 20 is a diagram showing the entire configuration of an image sensor100′ in the fifth embodiment, which is used in place of the image sensor100 shown in FIG. 2 described in the first embodiment. It is to be notedthat the same reference numerals are given to the same configurations asthose in FIG. 2, and a description thereof will be omitted. Further,since the configuration other than the image sensor 100′ is the same asthat of the first embodiment described above, description thereof isomitted.

A signal processing block 2000 includes a correction unit 207, a framememory 2001, and an addition unit 2002. The frame memory 2001 is astorage circuit which includes a temporary memory region 2001 a and anintegral memory region 2001 b and acquires and holds digital pixelsignals from each pixel during the exposure period. The temporary memoryarea 2001 a acquires digital pixel signals from respective pixels andtemporarily holds them. The pixel signals held in the temporary memoryarea 2001 a are input to the correction unit 207, and the samecorrection processes as those in the first embodiment is performed.

The addition unit 2002 adds the image signal newly read out from eachpixel and corrected to the image signal held in the integral memoryregion 2001 b for each identical address. The added image signal is heldagain in the integral memory region 2001 b. As a result, the integralmemory region 2001 b holds the pixel signal obtained by integratingpixel signals corrected by the correction unit 207 for each identicaladdress during the exposure period. Here, the temporary memory region2001 a has the same bit width per pixel as the digital pixel signaloutput from the unit pixel 201. On the other hand, the integral memoryregion 2001 b has a sufficiently large bit width per pixel as comparedwith the digital pixel signal output from the unit pixel 201.

The pixel signal held in the integral memory region 2001 b is output tothe outside of the image sensor 100 via the digital output unit 208after the end of the exposure period. Note that the frame memory 2001,the addition unit 2002, and the correction unit 207 may be providedoutside the image sensor 100′.

FIG. 21 shows an example of a chip layout of the image sensor 100′ inthe fifth embodiment. FIG. 21 corresponds to FIG. 6 of the firstembodiment, and the same components as those in FIG. 6 are denoted bythe same reference numerals, and description thereof will be omitted.

The image sensor 100′ has a structure in which a light receiving unitsubstrate 601, a counting unit substrate 602, and a frame memorysubstrate 2101 are stacked. Wirings of these substrates are electricallyconnected by using silicon penetrating electrodes or the like. On theframe memory substrate 2101, the frame memory 2001, the addition unit2002, the correction unit 207, and the digital output unit 208 arearranged. If the frame memory substrate 2101 is manufactured with afiner process than the light receiving unit substrate 601 and thecounting unit substrate 602, the frame memory 2001 can secure a largenumber of bits. It should be noted that the image sensor 100′ may beformed on a single substrate instead of forming on the stackedsubstrates.

FIG. 22 is a drive timing chart of the image capturing apparatus whenobtaining an image of one frame in the fifth embodiment. FIG. 22corresponds to the timing chart shown in FIG. 7 of the first embodiment,and processes that perform the same operations as those in FIG. 7 aredenoted by the same reference numerals, and the description thereof willbe omitted as appropriate.

In FIG. 22, the exposure period T between time t702 and time t703 isdivided into four exposure periods (hereinafter referred to as “dividedexposure periods”) T1 to T4, and a pixel signal is transferred from eachpixel to the frame memory 2001 for each divided exposure period.

When the divided exposure period T1 elapses at time t2201, the PLATsignals supplied to all the pixels from the vertical control circuit 202simultaneously change from L to H to L. As a result, the count valuecounted by the counter circuit 306 of each pixel during the dividedexposure period T1 is held in the pixel memory 307 of each pixel as apixel signal. When the pixel signal is held in the pixel memory 307, thePRES signal goes high immediately and the count value of the countercircuit 306 of each pixel is reset to 0. When the PRES signal returns toL, the resetting of the counter circuit 306 end, and the counter circuit306 of each pixel starts counting according to the incident photonsagain.

From time t2202 to time t2203, the VCLK signal and the HCLK signal aresupplied from the TG 204 to the vertical control circuit 202 and thehorizontal control circuit 203, respectively. The pixels aresequentially selected by the vertical control circuit 202 and thehorizontal control circuit 203, and the pixel signals held in the pixelmemories 307 are sequentially held in the temporary memory region 2001 ain the frame memory 2001. Gain correction and offset correction areperformed on the pixel signals held in the temporary memory region 2001a in the correction unit 207, and the corrected pixel signals are heldin the temporary memory region 2001 a again. The correction processesperformed by the correction unit 207 here is the same as those of thefirst to fourth embodiments, and the explanation thereof is omitted.

Then, in the addition unit 2002, the corrected pixel signal held in thetemporary memory region 2001 a and the pixel signal of the same addressheld in the integral memory region 2001 b are added, and the added pixelsignal is again stored in the integral memory region 2001 b. Thecorrection processes and the addition process are performed in parallelwith the operation of transferring the pixel signals in the pixelmemories 307 of the respective pixels to the frame memory 2001.

In the first addition process performed from time t2202 to time t2204,since pixel signals are not held in the integral memory region 2001 b inthe frame memory 2001, the pixel signals corrected by the correctionunit 207 are directly input to the integral memory region 2001 b.

When the divided exposure period T2 elapses at time t2204, the PLATsignal again changes from L to H to L so that the pixel signal of eachpixel is held in the temporary memory region 2001 a of the frame memory2001, and undergoes correction processes and addition process in thesame manner as in the period from time t2201 to time t2203. Then, thepixel signal which has undergone the addition process is held again inthe integral memory region 2001 b. Thereafter, the same processes arerepeated each time the divided exposure period elapses.

Here, it is necessary to complete transferring of the pixel signal heldin the pixel memory 307 of each pixel to the temporary memory region2001 a, the correction processes in the correction unit 207, and theaddition process in the addition unit 2002, performed from time t2201 totime t2203, by time t2204. In other words, these processes needs to becompleted before the next divided exposure period elapses. Therefore, anoperation frequency at which processes are completed within this periodis set. Alternatively, the length and/or the number of division of thedivided exposure periods are adjusted so that the processes arecompleted before the next divided exposure period elapses.

At time t703, when the exposure period T for one frame ends, the PENsignal becomes L. As a result, the counter circuit 306 of each pixel isdisabled, and the count value of the counter circuit 306 does notincrease. Also, the supply of the reverse bias voltage Vbias to thelight receiving unit 301 is stopped, and the light receiving unit 301stops outputting the PLS signal. Then, at time t2206, the PLAT signalswitches from L to H to L, and the count value counted by the countercircuit 306 at the end of the exposure period is held in the pixelmemory 307. Then, the PRES signal becomes H, and the count value of thecounter circuit 306 is reset to 0.

Thereafter, from time t2207 to time t2208, the pixel signal of eachpixel is transferred to the temporary memory region 2001 a, as in theperiod from time t2202 to time t2203. Then, the pixel signals undergonethe correction processes and the addition process are held in theintegral memory region 2001 b. At this time, the pixel signals held inthe integral memory region 2001 b are signals obtained by integratingthe pixel signals acquired from the pixel memories 307 during theexposure period from time t702 to time t703. Each of the pixel signalsis a signal corresponding to the number of photons incident during theexposure period.

According to the fifth embodiment as described above, the count rate iscalculated and corrected for each divided exposure period. By doing so,compared with the first embodiment, even when the luminance of a subjectchanges during the exposure period, it is possible to suitably suppressimage quality deterioration caused by a difference in received lightamount per unit time. It should be noted that the number of division ofthe exposure period and the length of the divided exposure period in theabove fifth embodiment are merely examples, and the present invention isnot limited thereto.

Sixth Embodiment

Next, a sixth embodiment of the present invention will be described. Inthe fifth embodiment described above, the count value of the countercircuit is reset every time the pixel signal is read out to the framememory 2001 during the exposure period. In the sixth embodiment, aconfiguration in which the count value of the counter circuit is notreset during the exposure period will be described. The configuration inthe sixth embodiment can cope with a case where the subject luminancechanges during the exposure period, as in the case of the fifthembodiment.

FIG. 23 shows a configuration of a signal processing block 2300 thatperforms signal processing on pixel signals output from the pixel area200. The signal processing block 2300 is used in place of the signalprocessing block 2000 shown in FIG. 20 described in the fifthembodiment. Since other parts are the same as those in the fifthembodiment, the description thereof will be omitted.

A frame memory 2301 is a storage circuit that acquires and holds adigital pixel signal from each pixel during the exposure period, and iscomposed of a temporary memory region A 2311, a temporary memory regionB 2312, and an integral memory region 2313. The temporary memory regionA 2311 and the temporary memory region B 2312 are temporary memories fortemporarily holding a digital pixel signal from each pixel. Every time areading operation of each pixel is performed during the exposure period,the read pixel signal is held in the temporary memory region A 2311 orthe temporary memory region B 2312 alternately. The integral memoryregion 2313 holds a value obtained by integrating the pixel signalscorrected by the correction unit 207 during the exposure period.

A subtraction unit 2302 calculates differences between the pixel signalsheld in the temporary memory region A 2311 and the pixel signals held inthe temporary memory region B 2312 for each address, and outputsdifference signals (difference values) to the correction unit 207. Sincethe correction processes of the correction unit 207 are the same asthose described in the first to fourth embodiments, the descriptionthereof will be omitted here. The addition unit 2002 adds the imagesignal corrected by the correction unit 207 to the image signal held inthe integral memory region 2313 of the frame memory 2301 for eachaddress. The added image signal is stored again in the integral memoryregion 2313. The pixel signal held in the integral memory region 2313 isoutput to the outside of the image sensor 100′ via the digital outputunit 208 after the exposure period has elapsed. Note that the signalprocessing block 2300 may be provided outside the image sensor 100′.

FIG. 24 shows a drive timing chart of the image capturing apparatus whenobtaining an image of one frame in the sixth embodiment. FIG. 24corresponds to the timing chart shown in FIG. 22 of the fifthembodiment, and processes that perform the same operations as those inFIG. 22 are denoted by the same reference numerals, and descriptionthereof will be omitted as appropriate.

In FIG. 24, as in the fifth embodiment, the exposure period T from timet702 to time t703 is divided into four divided exposure periods T1 toT4, and a pixel signal is read from each pixel to the frame memory 2301for each divided exposure period. However, unlike the fifth embodiment,the count value of the counter circuit 306 is not reset after readingthe pixel signal to the frame memory 2301.

When the divided exposure period T1 elapses at time t2401, the PLATsignals supplied to all the pixels from the vertical control circuit 202simultaneously change from L to H to L. As a result, the count valuecounted by the counter circuit 306 of each pixel during the dividedexposure period T1 is held in the pixel memory 307 of each pixel as apixel signal. From time t2402 to t2403, the VCLK signal and the HCLKsignal are supplied from the TG 204 to the vertical control circuit 202and the horizontal control circuit 203, respectively. The pixels aresequentially selected by the vertical control circuit 202 and thehorizontal control circuit 203, and the pixel signals held in the pixelmemories 307 are sequentially held in the temporary memory region in theframe memory 2001.

The PMEM signal is a control signal for selecting which of the temporarymemory region A 2311 and the temporary memory region B 2312 the pixelsignal is to be held, and is sent from the TG 204 to the frame memory2301. When the PMEM signal is at the L level, the pixel signal outputfrom the pixel area 200 is held in the temporary memory region A 2311and is held in the temporary memory region B 2312 when the PMEM signalis at the H level. In the period from time t2402 to time t2403, sincethe PMEM signal is at the L level, the pixel signals output from thepixel area 200 are held in the temporary memory region A 2311.

When the pixel signals are held in the temporary memory region A 2311,the subtraction unit 2302 subtracts the pixel signals held in thetemporary memory region B 2312 from the pixel signals held in thetemporary memory region A 2311 for each address to generate differencesignals. Then, the difference signals are output to the correction unit207. At this time, since no pixel signal is held in the temporary memoryregion B 2312, the pixel signals held in the temporary memory region A2311 are outputted to the correction unit 207. Here, the differencesignals output to the correction unit 207 are signals counted by thecounter circuits 306 during the divided exposure period T1. After thegain correction and the offset correction are performed on the pixelsignals input to the correction unit 207, the addition unit 2002 addsthe pixel signals after the correction processes and the pixel signalsheld in the integral memory region 2313 for each address. Then, theadded pixel signals are held again in the integral memory region 2313.The above difference process, correction processes and addition processare performed in parallel with the operation of transferring the pixelsignal of each pixel to the temporary memory region A 2311 of the framememory 2301. In addition, in the first addition process performed fromtime t2402 to time t2404, since pixel signals are not held in theintegral memory region 2313, the pixel signals corrected by thecorrection unit 207 are directly stored in the integral memory region2313.

Since the counter circuits 306 are not reset after holding the countvalues of the counter circuits 306 in the pixel memories 307 at timet2401, the count value continues to increase according to the number ofincident photons.

When the divided exposure period T2 elapses at time t2404, the PLATsignal again changes from L to H to L so that the pixel signal of eachpixel is output to the frame memory 2301 and held there, as in theperiod from time t2401 to time t2403. At this time, since the PMEMsignal becomes H, the pixel signal of each pixel is held in thetemporary memory region B 2312. The subtraction unit 2302 subtracts thepixel signals held in the temporary memory region A 2311 during timet2402 and time t2403 from the pixel signals held in the temporary memoryregion B 2312 for each address, and outputs the difference signals tothe correction unit 207. Here, the subtraction process performed by thesubtraction unit 2302 is to subtract the previously held signals fromthe newly held signals between the signals held in the temporary memoryregion A 2311 and the temporary memory region B 2312. Therefore, thedifference signals generated by this subtraction process are signalscorresponding to the numbers of photons incident during the dividedexposure period T2. The difference signals obtained by subtraction unit2302 undergo the correction processes and the addition process by thecorrection unit 207 and the addition unit 2002. Then, the pixel signalsundergone the addition process is stored again in the integral memoryregion 2313. After that, the same processes are repeated for eachdivided exposure period.

When the exposure period T ends at time t703, the PEN signal becomes L.As a result, the counter circuit 306 of each unit pixel 201 is disabled,and even if the PLS signal is input to the counter circuit 306, thecount value does not increase. Also, the supply of the reverse biasvoltage Vbias to the light receiving unit 301 is stopped, and the lightreceiving unit 301 stops outputting the PLS signal.

Then, at time t2406, the PLAT signal switches from L to H to L, and thecount value counted by the counter circuit 306 at the end of theexposure period is held in the pixel memory 307. Then, the PRES signalbecomes H, and the count value of the counter circuit 306 is reset to 0.

Thereafter, from time t2407 to time t2408, the pixel signal of eachpixel is transferred to either of the temporary memory region A 2311 orthe temporary memory region B 2312, as in the period from time t2402 totime t2403. Then, the pixel signals undergone the subtraction process,the correction processes and the addition process are held in theintegral memory region 2313. At this time, the pixel signals held in theintegral memory region 2313 are signals obtained by integrating thepixel signals acquired from the pixel memories 307 during the exposureperiod from time t702 to time t703. Each of the pixel signals is asignal corresponding to the number of photons incident during theexposure period.

According to the sixth embodiment as described above, the count rate iscalculated and corrected for each divided exposure period. By doing so,compared with the first embodiment, even when the luminance of a subjectchanges during the exposure period, it is possible to suitably suppressimage quality deterioration caused by a difference in received lightamount per unit time. It should be noted that the number of division ofthe exposure period and the length of the divided exposure period aremerely examples, and the present invention is not limited thereto.

Although the embodiments of the present invention have been describedabove, the present invention is not limited to these embodiments.Further, the above-described embodiments may be appropriately combined.

Other Embodiments

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully asanon-transitory computer-readable storage medium') to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc(BD)TM), a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2017-241119, filed on Dec. 15, 2017, which is hereby incorporated byreference herein in its entirety.

1-23. (canceled)
 24. An image sensor comprising one or more processorsand/or circuitry which functions as: a plurality of pixels each of whichdetects photons incident during a predetermined exposure period, countsa number of the photons, and outputs a first count value; a correctorthat acquires a correction coefficient based on a photometry resultobtained from a photometer and the first count value and corrects adetection error of the first count value using the correctioncoefficient, wherein the corrector acquires a larger value as thecorrection coefficient in a case where a luminance value based on thephotometry result is a first luminance value than in a case where theluminance value is a second luminance value which is smaller than thefirst luminance value.
 25. The image sensor according to claim 24,wherein each of the plurality of pixels has a detector that detectsphotons, a counter that counts a number of the detected photons andoutputs a counted value, and a memory that holds the counted value, andthe counted value held in the memory is output as the first count value.26. The image sensor according to claim 25, further comprising one ormore processors and/or circuitry which functions as an adder that, for apredetermined number of consecutive exposure periods, adds the firstcount values each of which is output each time the exposure periodelapses and is corrected by the corrector pixel by pixel, wherein thecounter resets the counted value each time the counted value is held inthe memory.
 27. The image sensor according to claim 25, wherein thedetector includes an avalanche photodiode and a quenching resistor forapplying a reverse bias voltage across the avalanche photodiode.
 28. Theimage sensor according to claim 25, wherein the detector includes anavalanche photodiode and a MOS transistor for applying a reverse biasvoltage across the avalanche photodiode.
 29. The image sensor accordingto claim 25, wherein the image sensor has a stacked structure, and thedetector and the counter are formed on different layers.
 30. The imagesensor according to claim 24, wherein the correction coefficient is avalue based on a difference between the second count value and a valueper unit time of an ideal count value in a case where no detection erroroccurs.
 31. The image sensor according to claim 24, further comprisingone or more processors and/or circuitry which functions as a secondmemory that stores correction coefficients corresponding to differentcounted values per unit time, wherein the corrector selects thecorrection coefficient corresponding to the second count value from thecorrection coefficients stored in the second memory.
 32. The imagesensor according to claim 24, wherein the corrector obtains thecorrection coefficient corresponding to the second count value using anapproximation function for obtaining a correction coefficientcorresponding to the counted value per unit time.
 33. The image sensoraccording to claim 24, wherein a part of the plurality of pixels areoptical black pixels which are shielded from light, and the image sensorfurther comprises a black level corrector that corrects the first countvalue output from a pixel which is not shielded from light and correctedby the corrector by using a value obtained by acquiring a secondcorrection coefficient for correcting a black level based on the secondcount value of the pixel which is not shielded from light and correctsan average value of the first count values output from the optical blackpixels and corrected by the corrector using the second correctioncoefficient.
 34. The image sensor according to claim 33, wherein theblack level corrector acquires the second correction coefficient basedon an average of the second count values of a pixel to be corrected andits surrounding pixels.
 35. The image sensor according to claim 33,wherein the black level corrector acquires the second correctioncoefficient based on an average of the second count values of a pixel tobe corrected respectively obtained in a predetermined number ofconsecutive exposure periods.
 36. An image capturing apparatuscomprising one or more processors and/or circuitry which functions as:an image sensor having a plurality of pixels each of which detectsphotons incident during a predetermined exposure period, counts a numberof the photons, and outputs a first count value; a photometer; acorrector that acquires a correction coefficient based on a photometryresult obtained from the photometer and the first count value andcorrects a detection error of the first count value using the correctioncoefficient, wherein the corrector acquires a larger value as thecorrection coefficient in a case where a luminance value based on thephotometry result is a first luminance value than in a case where theluminance value is a second luminance value which is smaller than thefirst luminance value.
 37. The image capturing apparatus according toclaim 36, wherein each of the plurality of pixels has a detector thatdetects photons, a counter that counts a number of the detected photonsand outputs a counted value, and a memory that holds the counted value,and the image capturing apparatus further comprises one or moreprocessors and/or circuitry which functions as an adder that, for apredetermined number of consecutive exposure periods, adds the firstcount values each of which is output each time the exposure periodelapses and is corrected by the corrector pixel by pixel, wherein thecounter resets the counted value each time the counted value is held inthe memory.
 38. The image capturing apparatus according to claim 36,wherein a part of the plurality of pixels are optical black pixels whichare shielded from light, and the image capturing apparatus furthercomprises one or more processors and/or circuitry which functions as ablack level corrector that corrects the first count value output from apixel which is not shielded from light and corrected by the corrector byusing a value obtained by acquiring a second correction coefficient forcorrecting a black level based on the second count value of the pixelwhich is not shielded from light and correcting an average value of thefirst count values output from the optical black pixels and corrected bythe corrector using the second correction coefficient.
 39. An imageprocessing method comprising: calculating a second count value per unittime based on a first count value obtained from an image sensor having aplurality of pixels each of which detects photons incident during apredetermined exposure period and outputs a counted value of a number ofthe detected photons, and the exposure period; and acquiring acorrection coefficient based on a photometry result obtained from aphotometer and the first count value and correcting a detection error ofthe first count value using the correction coefficient, wherein a largervalue is acquired as the correction coefficient in a case where aluminance value based on the photometry result is a first luminancevalue than in a case where the luminance value is a second luminancevalue which is smaller than the first luminance value.
 40. The imageprocessing method according to claim 39, wherein each of the pluralityof pixels has a detector that detects photons, a counter that counts anumber of the detected photons and outputs a counted value, and a memoryholds the counted value, and the image processing method furthercomprises, for a predetermined number of consecutive exposure periods,adding the corrected first count values each of which is output eachtime the exposure period elapses pixel by pixel, wherein the counterresets the counted value each time the counted value is held in thememory.
 41. The image processing method according to claim 39, wherein apart of the plurality of pixels are optical black pixels which areshielded from light, and the image processing method further comprisescorrecting the first count value output from a pixel which is notshielded from light and corrected by the corrector by using a valueobtained by acquiring a second correction coefficient for correcting ablack level based on the second count value of the pixel which is notshielded from light and correcting an average value of the first countvalues output from the optical black pixels and corrected by thecorrector using the second correction coefficient.
 42. A non-transitorystorage medium readable by a computer, the storage medium storing aprogram that is executable by the computer, wherein the program includesprogram code for causing the computer to perform the image processingmethod that comprises: calculating a second count value per unit timebased on a first count value obtained from an image sensor having aplurality of pixels each of which detects photons incident during apredetermined exposure period and outputs a counted value of a number ofthe detected photons, and the exposure period; and acquiring acorrection coefficient based on a photometry result obtained from aphotometer and the first count value and correcting a detection error ofthe first count value using the correction coefficient, wherein a largervalue is acquired as the correction coefficient in a case where aluminance value based on the photometry result is a first luminancevalue than in a case where the luminance value is a second luminancevalue which is smaller than the first luminance value.